Web Image Context Extraction: Methoden und Evaluation

Images on the Web come in hand with valuable textual content on hosting web pages that can be exploited to generate image annotations. However, web documents are usually composed of contents to multiple topics and the context of an image makes only a small portion of the full text of the web page. In order to get qualitative descriptions, methods that are able to extract the image context become essential. Existing solutions in the literature reach from simple full text extractors to intelligent approaches that perform a page segmentation as a preprocessing step. To be able to evaluate and compare the different methods, we introduce an evaluation framework that includes a ground truth dataset consisting of twelve different testing collections. The accordance between extraction output and ground truth is estimated using newly adapted evaluation measures that are a part of the framework. Most of the existing methods are based on simple heuristics and hence in general can not deal with the variety of different web page designs. Our first approach is therefore more adaptive: it arranges first the smallest content units of a web page to possible context candidates (articles) and assigns then to each image of the web page the most suitable candidate. This approach is extended by concepts that are able to handle the two-dimensional HTML-tables that are frequently used as layout elements. Another contribution is an image context extraction method that is based on page segmentation as a preprocessing step. By separating a web page into blocks of coherent topics, the images just can be associated with the complete text of the common block. In an extended analysis, we investigate different approaches to solve the page segmentation task by web content clustering. Different representations for web contents are combined with various clustering approaches and evaluated. The gained experience is used to build a novel clustering-based context extraction method. Both methods achieve very good results on almost all test collections and can thus be applied as a preprocessing step in applications that can benefit from images with descriptions.